Search results

Rankings have become a popular topic in the social sciences over the past two decades. Adding to these debates, the present volume assembles studies that explore a variety of empirical settings, emphasizing the importance of acknowledging that there are multiple “Worlds of Rankings.” To this end, the first part of the chapter addresses the implications of two modes of criticism that characterize much of the scholarly work on rankings and summarizes extant conceptual debates. Taking stock of what we know, the second part distinguishes three areas of empirical research. The first area concerns the activities of those who produce rankings, such as the collection of data or different business strategies. Studies in the second area focus on inter-organizational, field-level, or discursive phenomena, particularly how rankings are received, interpreted, and institutionalized. The third area covers the manifold effects that research has unveiled, ranging from the diffusion of practices and changes in organizational identities to emotional distress. Taken together, the contributions to this volume expand our knowledge in all three areas, inviting new debates and suggesting pathways forward.

Organizational metrics – including rankings, ratings, and other forms of public assessment – are inextricably tied to uncertainty. Metrics are not only responses to uncertainty in the organizational environment, but they also create new forms of uncertainty within the organizations they evaluate. Given this, it is productive to consider these metrics in relation to the garbage can model of organizational decision making, a framework that was designed to provide insight into uncertain and ambiguous contexts. In this paper, the authors use the case of patient experience surveys to argue for the value of this model for understanding responses to metrics in particular conditions. Specifically, the authors demonstrate how the different features of the garbage can model manifest themselves within organizations managing numbers, and the authors then use these findings to discuss the measurement conditions that promote garbage can responses, the distinctive types of unintended consequences these responses might produce, and the implications of the garbage can model for the understanding of metrics more generally.

The thesis that rankings do more than just make visible an organization’s position viz-á-viz a competitor, but stimulate new competitive rivalries, has provoked much interest. Yet, to date, scholars lack an understanding of how such competitive rivalries unfold at the level of organizational strategy. Put simply, if competition is played out in rankings, how does this change the way organizations strategize? We answer this question through an ethnographic study of how information technology organizations engage with rankings. The strategic responses we observed included “leapfrogging a rival,” “de-positioning a competitor,” “owning a market,” and “encouraging a breakout,” which together are theorized as “ranking strategy.” This novel conceptualization extends understanding of the organizational response to rankings by showing how common reactions like gaming are only the tip of the iceberg of a broader array of strategic responses. The study also throws light on the different ways a ranking can pattern competitive rivalries, including creating more episodic forms of rivalry.

The statement that evaluation works differently in different contexts might seem fairly obvious, but given how central it is to virtually all aspects of modern life, it is important to understand how these differences affect the objects of evaluation and the people doing the evaluation. Drawing on a mixed-methods study of evaluation in figure skating and classical music, the author addresses how evaluation practices affect judging and performance. In the early- to mid-2000s, the figure skating world transitioned from a judging system where judges used two overall marks to rank skaters to a much more rigid system where judges rate very specific aspects of performances without actively ranking them. These changes have impacted judges and skaters. In the classical music competitions the author focuses on, judges use deliberations to rank performances. The relative flexibility of these evaluation practices generally does not affect judges or performances as much. Building on research on the effects of measurement and evaluation systems, the author argues that formal, specific rules surrounding evaluation shape judging and performance more than informal, diffuse rules. Focusing on competition settings in skating and music, the author discusses how evaluation practices affect program and repertoire construction and the technical and artistic aspects of judging and performance in these fields. In addition to supplementing research suggesting that evaluation systems actively shape what they are designed to evaluate, this work highlights how different types of evaluation contribute to different responses in terms of the evaluation itself and the objects of evaluation.

Performance metrics have become widely used and much lamented – about tools for measuring healthcare quality. In this paper, the authors reflect on the development and use of performance metrics in healthcare regulation and clinical practice. Studying multi-actor settings of performance measurement systems in healthcare in Sweden and the Netherlands, the authors show how regulatory agencies (i.e., the inspectorate and national registries), patients, hospitals, and practitioners engage in the constitution of healthcare practices through developing performance indicators that form the input for ranking, ensuing intensive dialogues on what should be measured and accounted for, and to what effects. The authors analyze this process as caring for numbers. The authors discern two practices of caring for numbers: validating and contexting. Validating refers to the practices of making numbers reflect those practices they intend to depict; contexting is about how with the use of numbers specific contexts of healthcare are built. These processes together emphasize the performative character of numbers as well as the reflexive uses of performativity. The paper shows how collaborative and rather pragmatic practices of caring for numbers co-construct specific practices of healthcare. Though this reflexive entanglement of production and use of numbers actors not only constitute specific performance metrics and ranking practices but also perform healthcare.

The visualization of ranking information in global public policy is moving away from traditional “league table” formats and toward dashboards and interactive data displays. This paper explores the rhetoric underpinning the visualization of ranking information in such interactive formats, the purpose of which is to encourage country participation in reporting on the Sustainable Development Goals. The paper unpacks the strategies that the visualization experts adopt in the measurement of global poverty and wellbeing, focusing on a variety of interactive ranking visualizations produced by the OECD, the World Bank, the Gates Foundation and the ‘Our World in Data’ group at the University of Oxford. Building on visual and discourse analysis, the study details how the politically and ethically sensitive nature of global public policy, coupled with the pressures for “decolonizing” development, influence how rankings are visualized. The study makes two contributions to the literature on rankings. First, it details the move away from league table formats toward multivocal interactive layouts that seek to mitigate the competitive and potentially dysfunctional pressures of the display of “winners and losers.” Second, it theorizes ranking visualizations in global public policy as “alignment devices” that entice country buy-in and seek to align actors around common global agendas.

Governance initiatives based on rankings are predicated on the possibility of making companies compete for the achievement of social goals by means of public comparisons of performance. The public of such performance – the ranker and various stakeholders in whose name the ranker speaks – thus fulfills the role of a “third party” whose favor is sought by competitors in what Simmel analyzed as “pure” or “indirect” competition. Yet little is known about how rankers seek to produce or entice such favor in order to enact competition. Through the case of the Access to Medicine Index, we examine the process of selective foregrounding, enticing and orchestration of different stakeholders through the gearing of the ranking’s information infrastructure aimed at optimizing the type and intensity of the competitive pressure exercised on the ranked. We illustrate how the ranker segments the public into different third parties, some well-identified stakeholders alongside a more anonymous audience. We find that stakeholders perceived as wielding legitimate power in the eyes of companies (such as investors) are actively equipped with the tools to witness competition, whereas stakeholders seen as powerful but involved in an agonistic relation with the companies (such as radical Non-Governmental Organizations) are discretely groomed at a distance, while those stakeholders with no perceived power over companies tend to remain unequipped. Whilst the gaze of stakeholders as third parties is differentiated along the lines of a hierarchy of observations, the voice of stakeholders as representatives of different interests is equalized and unified so as to adhere to an ideal of consensus. We reflect on how the needs of competition and those of stakeholder representation come to intersect in the particular governance space of access to medicine. Competition, far from being the automatic consequence of rankings, emerges as a contrived and laborious enactment requiring painstaking attention to publics and their selective equipment as third parties. Understanding the modes of such enactment is thus crucial for appreciating rankings’ governance outcomes.

Rankings are widely regarded as particularly well-suited for capturing the public eye, which is considered a reason why they have become ubiquitous. However, we know little about how rankings direct media attention, as well as how media in turn shape and help sustain careers of specific rankings in the public over longer periods of time. To advance our understanding of the discursive dynamics at the intersection of rankings and the press, this study examines the media career of the Global Slavery Index (GSI) by analyzing 361 newspaper and magazine articles, published between the release of index’s inaugural edition in 2013 and until the end of 2019. To interpret the media coverage, the study draws attention to GSI’s universality, highly rationalized character, and a pledge to spotlight violation of the global moral order. The examination of the media coverage points to the following properties of the index as having shaped and helped sustain its career in the public: (1) repeated publication; (2) broad conceptualization of modern slavery; and (3) the construction thereof as a measurable global burden. The study finds that, throughout the period, the media were remarkably consistent in amplifying the most dramatic elements of the index. Over time, however, the index was increasingly more invoked for other purposes, usually either to lend credibility to a story or as a way of embedding local and situational concerns into global narratives.

Extant research conceptualizes rankings as measures that fundamentally shape modern life by virtue of being publicly available. Yet, studies seldom explore the act of publishing when accounting for the attention rankings raise in larger stakeholder audiences. In short, we know a lot about the impact of rankings, but considerably less about the organizational practices devised by those who produce them – the rankers. Borrowing from Goffman, the paper conceptualizes modern rankings as public performances carefully prepared on backstages and unfolding on multiple frontstages. Using a qualitative data set, the paper traces the full spectrum of organizational practices that make rankings public performances: on the backstage, launch dates have to be set, numbers packaged, and “teams” prepared; on the frontstage, performances are held through face-to-face interactions (at launch events) as well as in a variety of mediated settings. Overall, the findings indicate that the more ranking organizations seek the attention of larger stakeholder audiences, the more the publication process is transformed into what one of the informants describes as “a big firework.”